Investigating Rumor News Using Agreement-Aware Search

In recent years, rumor news has been generated by humans as well as robots in order to attract readership, in uence opinions, and increase internet click revenue. Its detrimental e ects have become a worldwide phenomenon, leading to confusion over facts and causing mistrust about media reports. However, evaluating the veracity of news stories can be a complex and cumbersome task, even for experts. One of the challenging problems in this context is to automatically understand di erent points of view, i.e., whether other news articles reporting on the same problem agree or disagree with the reference story. This can then lead to the identi cation of news articles that propagate false rumors (a.k.a., “fake news”). In this paper, we propose a novel agreement-aware search framework, Maester, for dealing with the problem of rumor detection. Given an investigative question summarizing some news story or topic, Maester will retrieve related articles to that question, assign and display top articles from agree, disagree, and discuss categories to users, and thus provide a more holistic view. Our work makes two technical observations. First, relatedness can commonly be determined by keywords and entities occurred in both questions and articles. Second, the level of agreement between the investigative question and the related news article can often be decided by a few key sentences. Accordingly, we design our approach for relatedness detection to focus on keyword/entity matching using gradient boosting trees, while leveraging recurrent neural networks and posing attentions to key sentences to infer the level of agreement. Our evaluation is based on a recently published dataset from the Fake News Challenge (FNC) “stance detection” task. Extensive experiments demonstrate up to an order of magnitude improvement of Maester over all baseline methods, including the FNC winning solution, for agreement-aware search as well as slightly improved accuracy based on the same metrics used in FNC.